Combinatory Categorial Grammar Parsing with NLTK

(C) 2019 by Damir Cavar

This is a tutorial related to the discussion of grammar engineering and parsing in the class Alternative Syntactic Theories and Advanced Natural Language Processing taught at Indiana University in Spring 2019.

The examples presuppose an installed Python 3.x NLTK module with all the dependent modules and packages, as well as the data set for NLTK.

Introduction

coming soon...

Code Examples

The initial code examples here were taken from http://www.nltk.org/howto/ccg.html and adapted for our course needs.


In [5]:
from nltk.ccg import chart, lexicon

We can specify a lexicon as follows:


In [6]:
lex = lexicon.fromstring('''
:- S, NP, N, VP
Det :: NP/N
Pro :: NP
Modal :: S\\NP/VP

TV :: VP/NP
DTV :: TV/NP

the => Det

that => Det
that => NP

I => Pro
you => Pro
we => Pro

chef => N
cake => N
children => N
dough => N

will => Modal
should => Modal
might => Modal
must => Modal

and => var\\.,var/.,var

to => VP[to]/VP

without => (VP\\VP)/VP[ing]

be => TV
cook => TV
eat => TV

cooking => VP[ing]/NP

give => DTV

is => (S\\NP)/NP
prefer => (S\\NP)/NP

which => (N\\N)/(S/NP)

persuade => (VP/VP[to])/NP
''')

We instantiate a parser instance using this lexicon specification:


In [7]:
parser = chart.CCGChartParser(lex, chart.DefaultRuleSet)

The following function wraps the parser calls. It takes a parser object as the first argument and the sentence as a string as the second.


In [8]:
def parse(myparser, sentence):
    for p in myparser.parse(sentence.split()):  # doctest: +SKIP
        chart.printCCGDerivation(p)
        break

In [9]:
parse(parser, "you prefer that cake")


 you    prefer      that   cake
 NP   ((S\NP)/NP)  (NP/N)   N
                  -------------->
                        NP
     --------------------------->
               (S\NP)
--------------------------------<
               S

In [10]:
parse(parser, "that is the cake which you prefer")


 that      is        the    cake      which       you    prefer
  NP   ((S\NP)/NP)  (NP/N)   N    ((N\N)/(S/NP))  NP   ((S\NP)/NP)
                                                 ----->T
                                              (S/(S\NP))
                                                 ------------------>B
                                                       (S/NP)
                                 ---------------------------------->
                                               (N\N)
                           ----------------------------------------<
                                              N
                   ------------------------------------------------>
                                          NP
      ------------------------------------------------------------->
                                 (S\NP)
-------------------------------------------------------------------<
                                 S

In [11]:
nosub_parser = chart.CCGChartParser(lex, chart.ApplicationRuleSet + chart.CompositionRuleSet + chart.TypeRaiseRuleSet)

In [12]:
parse(nosub_parser, "that is the dough which you will eat without cooking")

In [13]:
parse(parser, "that is the dough which you will eat without cooking")


 that      is        the    dough      which       you     will        eat          without           cooking
  NP   ((S\NP)/NP)  (NP/N)    N    ((N\N)/(S/NP))  NP   ((S\NP)/VP)  (VP/NP)  ((VP\VP)/VP['ing'])  (VP['ing']/NP)
                                                  ----->T
                                               (S/(S\NP))
                                                                             ------------------------------------->B
                                                                                         ((VP\VP)/NP)
                                                                    ----------------------------------------------<Sx
                                                                                       (VP/NP)
                                                       ----------------------------------------------------------->B
                                                                               ((S\NP)/NP)
                                                  ---------------------------------------------------------------->B
                                                                               (S/NP)
                                  -------------------------------------------------------------------------------->
                                                                       (N\N)
                           ---------------------------------------------------------------------------------------<
                                                                      N
                   ----------------------------------------------------------------------------------------------->
                                                                 NP
      ------------------------------------------------------------------------------------------------------------>
                                                         (S\NP)
------------------------------------------------------------------------------------------------------------------<
                                                        S

In [14]:
parse(parser, "that is the cake which we will persuade the chef to cook")


 that      is        the    cake      which       we     will           persuade        the    chef       to         cook
  NP   ((S\NP)/NP)  (NP/N)   N    ((N\N)/(S/NP))  NP  ((S\NP)/VP)  ((VP/VP['to'])/NP)  (NP/N)   N    (VP['to']/VP)  (VP/NP)
                                                 ---->T
                                              (S/(S\NP))
                                                                                      -------------->
                                                                                            NP
                                                                  ---------------------------------->
                                                                            (VP/VP['to'])
                                                                                                    ------------------------>B
                                                                                                         (VP['to']/NP)
                                                                  ---------------------------------------------------------->B
                                                                                           (VP/NP)
                                                     ----------------------------------------------------------------------->B
                                                                                   ((S\NP)/NP)
                                                 --------------------------------------------------------------------------->B
                                                                                   (S/NP)
                                 ------------------------------------------------------------------------------------------->
                                                                            (N\N)
                           -------------------------------------------------------------------------------------------------<
                                                                           N
                   --------------------------------------------------------------------------------------------------------->
                                                                      NP
      ---------------------------------------------------------------------------------------------------------------------->
                                                              (S\NP)
----------------------------------------------------------------------------------------------------------------------------<
                                                             S

In [15]:
parse(parser, "that is the cake which we will persuade the chef to give the children")


 that      is        the    cake      which       we     will           persuade        the    chef       to            give       the    children
  NP   ((S\NP)/NP)  (NP/N)   N    ((N\N)/(S/NP))  NP  ((S\NP)/VP)  ((VP/VP['to'])/NP)  (NP/N)   N    (VP['to']/VP)  ((VP/NP)/NP)  (NP/N)     N
                                                 ---->T
                                              (S/(S\NP))
                                                                                      -------------->
                                                                                            NP
                                                                  ---------------------------------->
                                                                            (VP/VP['to'])
                                                                                                                                 ------------------>
                                                                                                                                         NP
                                                                                                                   -------------------------------->
                                                                                                                               (VP/NP)
                                                                                                    ----------------------------------------------->B
                                                                                                                     (VP['to']/NP)
                                                                  --------------------------------------------------------------------------------->B
                                                                                                       (VP/NP)
                                                     ---------------------------------------------------------------------------------------------->B
                                                                                              ((S\NP)/NP)
                                                 -------------------------------------------------------------------------------------------------->B
                                                                                               (S/NP)
                                 ------------------------------------------------------------------------------------------------------------------>
                                                                                       (N\N)
                           ------------------------------------------------------------------------------------------------------------------------<
                                                                                      N
                   -------------------------------------------------------------------------------------------------------------------------------->
                                                                                  NP
      --------------------------------------------------------------------------------------------------------------------------------------------->
                                                                         (S\NP)
---------------------------------------------------------------------------------------------------------------------------------------------------<
                                                                         S

In [ ]: